Input/output interface for computer aided diagnosis (CAD) system

ABSTRACT

A user interface for facilitating a computer-aided diagnosis (CAD) includes an image input device that receives and scans the multiple images, and a film feeder that transports films from a film developer to the scanner. A multi-image identification input device uses bar code reading, numeric keypad, keyboard, network, and/or mouse to enter multiple image identification prior to image scanning. A display includes an image identification area, case processing status area, and a CAD detection and user modification area. The user can edit CAD results—accept, remove—and can add new diagnosis results to create a new composite diagnosis result.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 10/414,292, filed on Apr. 16, 2003, commonly-assigned, havingthe same title as this application, and which claims the benefit of thepriority date of U.S. Provisional Application No. 60/394,239, filed onJul. 9, 2002. Both of these applications are incorporated herein byreference in their entireties.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to methods and systems for the computeraided diagnosis (CAD) of abnormalities and digital processing ofradiological images. More specifically, the present invention relates toa user interface for an automated method and system for the re-screeningand detection of abnormalities.

2. Background Art

Computer-aided diagnosis (CAD) systems use digital processing methods toassist users in the identification of abnormalities. U.S. Pat. No.5,917,929 describes a user interface for facilitating the input of filmsinto a CAD system. The user interface includes a scanner that receivesand scans the film-based images and a film feeder that holds andtransports them to the scanner. A touch-sensitive display screen isprovided to display status information to and receive instructions froman operator. The display screen displays miniature images of the filmshaving color-coded borders, the colors indicating to the operator thestage of the processing of the films. The interface also includes a barcode reader and allows the operator to conduct procedures for testingthe scanner. However, there are a number of capabilities, in addition tothe above, that could be furnished, but are not.

SUMMARY OF THE INVENTION

The object of the current invention is to provide a more flexible userinterface for a CAD system with greater capabilities than in prior artuser interfaces. More specifically, the current invention has as itsobjects at least the following: (1) to allow the user to changediagnosis results from the output of the CAD system; (2) to allow theuser to manipulate images after the CAD processing; (3) to allow the CADprocessor to connect to an image acquisition device such as a filmdeveloper, film digitizer or phospher-plate scanner directly withouthuman interface; (4) to allow the CAD processor to connect to multipleimage acquisition devices; and (5) to allow the entry of multiple imageidentification for multiple image scanning.

In one embodiment, the present invention comprises a user interfaceincluding an input interface. The input interface may accommodate bothimage inputs and data inputs. To accommodate image input, the inputinterface may comprise at least one film feeder and at least one imageacquisition device. It may further comprise multiple image feeders andscanners. At least one of the image feeders and/or scanners may receiveinput directly from an image developer. The input interface may furthercomprise means by which to receive digital image inputs, including viawireless communications and via computer networks.

To accommodate data inputs, the input interface may comprise a keyboard,a keypad, a mouse, a bar code reader, and/or other input devices. It mayfurther include a voice interface and/or a writing interface (e.g., adigital tablet). Such data inputs may also be furnished via wirelessmeans or via computer networks.

In another embodiment, the invention comprises an output interfaceincluding display means. The display means may include small “thumbnail”displays of multiple images, and it may also include a large display ofa single image. The display may include indications of suspectedabnormalities detected by CAD processing. The display may furtheraccommodate user-identified user-accepted, and user-deletedabnormalities, entered using an input interface.

The output interface may further include displays of image data and ofcomments entered by a user. It may also include options and displays foruse in conjunction with an input interface.

Definitions

In describing the invention, the following definitions are applicablethroughout (including above).

A “computer” refers to any apparatus that is capable of accepting astructured input, processing the structured input according toprescribed rules, and producing results of the processing as output.Examples of a computer include a computer; a general-purpose computer; asupercomputer; a mainframe; a super mini-computer; a mini-computer; aworkstation; a microcomputer; a server; an interactive television; ahybrid combination of a computer and an interactive television; andapplication-specific hardware to emulate a computer and/or software. Acomputer can have a single processor or multiple processors, which canoperate in parallel and/or not in parallel. A computer also refers totwo or more computers connected together via a network for transmittingor receiving information between the computers. An example of such acomputer includes a distributed computer system for processinginformation via computers linked by a network.

A “computer-readable medium” refers to any storage device used forstoring data accessible by a computer. Examples of a computer-readablemedium include a magnetic hard disk; a floppy disk; an optical disk,like a CD-ROM or a DVD; a magnetic tape; a memory chip; and a carrierwave used to carry computer-readable electronic data, such as those usedin transmitting and receiving e-mail or in accessing a network.

“Software” refers to prescribed rules to operate a computer. Examples ofsoftware include software; code segments; instructions; computerprograms; and programmed logic.

A “computer system” refers to a system having a computer, where thecomputer comprises a computer-readable medium embodying software tooperate the computer.

A “network” refers to a number of computers and associated devices thatare connected by communication facilities. A network involves permanentconnections such as cables or temporary connections such as those madethrough telephone or other communication links, or both. Examples of anetwork include an internet, such as the Internet; an intranet; a localarea network (LAN); a wide area network (WAN); and a combination ofnetworks, such as an internet and an intranet.

A “computer-aided diagnosis (CAD) system” is a computer system that:

-   -   receives patients' diagnosis images such as patients' x-ray        images, pap smears, mammograms, or the like;    -   processes patients' diagnosis images; and    -   generates CAD results indicating either potential abnormalities        at specific locations (for example, the location of a disease)        in the patients' diagnosis images or potential abnormalities        appearing on entire patients' diagnosis images. (CAD usually        only indicates that the image may have some abnormality but does        not specify the specific location on the image.)

A physician or other health care provider can use the CAD results alongwith other tools and information to determine the specifics andlocations of diseases for the patients.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described in conjunction withthe accompanying drawings, in which:

FIG. 1 shows a general block diagram of a CAD system;

FIG. 2 shows a block diagram of a CAD system according to one embodimentof the invention, showing an exemplary embodiment of an input interface;

FIG. 3 shows a block diagram of a CAD system according to an embodimentof the invention;

FIG. 4 shows a screen shot demonstrating an embodiment of an interfacefor inputting image ID numbers using bar codes; and

FIGS. 5-8 show screen shots demonstrating embodiments of an outputinterface according to the invention.

DESCRIPTION OF EMBODIMENTS OF THE INVENTION

In this disclosure, a chest X-ray based CAD system is used as anexample. The invention is not, however, limited to such a system, but isapplicable to CAD systems, in general. Furthermore, the embodimentsdescribed here involve a single processing unit connecting with one ormultiple image acquisition devices; however, the invention is alsoapplicable to a configuration consisting of multiple processing units.

The invention comprises a method and system for providing interfaceswith a CAD system. Such interfaces include both input and outputinterfaces.

FIG. 1 shows a block diagram of a typical CAD system. The heart of thesystem is a computer system 2, which runs software implementing the CADfunctionality. Data from users and/or other systems enters computersystem 2 through input interface 1. Computer system 2 provides outputsthrough an output interface 3. While input interface 1 and outputinterface 3 are shown as separate blocks, they may be combined as asingle input/output interface.

Input interface 1 may comprise a number of different input devices.Computer system 2 may have the capability of interacting with only asingle input device; however, it will typically be capable ofinteracting with multiple input devices. In general, there are two typesof inputs: user input (e.g., alphanumeric data, bar code data,selections/instructions, marks of user-identified orother-devices-identified abnormalities, and the like) and image data(e.g., X-ray images, computer tomography (CT) images, and the like).Input interface 1 will generally include means for facilitating bothtypes of inputs.

FIG. 2 shows a block diagram of a computer system capable of receivinginput from multiple image input devices. In FIG. 2, computer system 2and output interface 3 are shown as a single block, labeled “2, 3”, forconvenience. As shown in FIG. 2, image input devices are connected tothe system 2, 3 and provide source interfaces with multiple imagesources. These devices may include single film scanners 11, 12 ormultiple film scanners 10, 12 or other image acquisition devices, whichwould include digitization if the images were not already in digitalform. For example, digital image data may be input directly from adigital image source 13. Such other image acquisition devices areintended to include, for example, means for receiving wirelesstransmissions of images and means for receiving images from a computernetwork. As shown, a multiple film scanner can be a single scanner 12with multi-film feeder 10, or several scanners 12, each with a singlefilm feeder 11, or multiple scanners 12, each with a multi-film feeder10.

There are several types of film scanners on the market. One exemplarytype is a laser scanner made by Lumysis (now part of Kodak). The Lumysisscanner emits laser light and uses a photo multiplier to acquire thesignal. Another type of scanner is a CCD scanner, like those produced byHowtek and Vidar. The Vidar scanner emits fluorescence light and uses aCCD to acquire the image, whereas the Howtek scanner emitsnarrow-bandwidth light and uses a CCD to acquire the image. The Vidarscanner, used in an exemplary system embodiment, can accept up totwenty-five films at a time in its feeder.

Yet another image input possibility for a film-based CAD system is for afilm scanner to be directly connected to a film processor. This is shownin FIG. 3. As shown, film developer 15 is connected to film feeder 14,which may be either a single film feeder or a multiple film feeder. Asin FIG. 2, film feeder 14 feeds the film(s) to a film scanner 12, whichprovides image input to the CAD system 2, 3. Thus, developed films arethereby automatically sent to the CAD system and processed.

Input interface 1 may further facilitate multiple image ID managementusing bar codes or data entered using a numeric keypad, keyboard,network (for example, over an Internet connection), or other inputdevices. FIG. 4 shows an exemplary implementation of the invention usinga bar code image ID input function (i.e., FIG. 4 is a screen shot ofwhat would be displayed to the user; the display itself is technicallypart of output interface 3). In this implementation, a user can use barcodes to input multiple image IDs prior to the film scanning (or otherimage input). The user may, alternatively, use a keyboard, keypad, orother means for inputting image IDs. During film scanning (image input)processing, each ID will be assigned to each digitized image in apre-sorted order or according to some other predetermined order, whichmay be determined by a user.

As discussed above, input interface 1 also generally includes devices tofacilitate user input. Such devices may include, for example, a numerickeypad, a keyboard, a mouse, an electronic pen, a touch screen (whichwould be a hybrid input interface 1/output interface 3), or a voiceinterface. Inputs made using such devices may be used for either or bothpurposes of inputting data about images and inputting informationnecessary to run the CAD system (e.g., telling the system to run,entering parameters, accepting and rejecting marks/indicationsassociated with abnormalities identified by the CAD system, adding a newmark/indication identified by the user, selecting images, etc.). In thecase of a voice interface, exemplary voice interfaces include avoice-activated device that could be used to operate the CAD systemand/or a voice recording device to record a physician's diagnosis orother comments.

Output interface 3 generally consists of all devices providing output tothe system user. Such devices may include monitors, touch screens, audiospeakers, or any other output devices. Output interface 3, together withinput interface 1 permits a user to initiate, control, and review andmanipulate results of CAD processing.

A particular embodiment of an output interface 3 of a CAD systemaccording to the present invention allows a user, in either a newdiagnosis result or result modified from the CAD result, to

-   -   Accept CAD results by keeping some or all CAD results;    -   Reject CAD results by removing some or all CAD results;    -   Add new diagnosis result(s);    -   Apply any combination of the above to generate a final diagnosis        result.        Particular embodiments of output interface 3 that support these        functions will be discussed below.

FIG. 5 shows a screen shot depicting exemplary features for implementingthe functions discussed above. FIG. 5 includes a mosaicarrangement/image management display 51 consisting of multiple thumbnailimages, where a display of image identification data is included foreach image. The individual thumbnail images may be implemented withmeans by which to indicate processing status to a user; this may be donein a number of ways. For example, in U.S. Pat. No. 5,917,929, a border52 of each thumbnail image takes on a different color at each stage ofprocessing. This could be similarly implemented using a dot that turns adifferent color to indicate each stage of processing. Alternatively, anumber of different colored dots could be used for each image, with eachcolor associated with a different stage of processing; in thisimplementation one dot, indicating the current stage of processing forthe image, would be brighter than the others (similar to colored lightsbeing turned on and off). In another embodiment, this function isimplemented by means of an alphanumeric display associated with eachthumbnail image, the alphanumeric display indicating the stage ofprocessing of the associated image; this may be in the form of astatement of the stage of processing or a code indicating the stage. Inyet another embodiment, the display indicating the stage of processingmay be in the form of a pictorial display for each thumbnail imageincluding one or more symbols indicating the stage of processing.

The thumbnail images in FIG. 5 may be replaced by a text-only displaysfor the images, in an alternative embodiment of the invention.

FIG. 5 also includes an image display 55 of a CAD result or other image.Image display 55 preferably comprises at least one-fourth of the totalavailable area of the screen, in order to provide the user with an imagesuitable for careful inspection. The image to be displayed in the imagedisplay 55 may be selected by a user. Image display 55 may also beoutfitted with scale indicators 57. Scale indicators 57 may be providedin units of inches, centimeters, etc.; in an exemplary embodiment of theinvention, the units are user-selected. In an exemplary embodiment ofthe invention, image display 55 may be capable of supporting variousoutput image resolutions (for example, 37.5, 75, 150, 300, 600 dpi);furthermore, the output image resolution may be user-selectable.

When the image displayed in the image display 55 represents a CADoutput, the results of CAD processing will generally be displayed on theimage. In FIG. 5, suspected abnormalities, as determined by a CADsystem, are shown surrounded by circles 56. Other shapes mayalternatively be used.

The screen shown in FIG. 5 also includes an information display field54. Information display field 54 may be used to display, for example,date, time, operator, patient ID number, or other similar information.In FIG. 5, information display field 54 shows an exemplary embodiment inwhich the information displayed is taken from a portion of an imageinput indicating such information as previously described.

The output interface 3 may also serve as part of input interface 1. InFIG. 5, a display of options 53 contains several different “buttons” forselection by a user, using an input device (e.g., mouse, electronic pen,etc.). In the exemplary embodiment shown, the user can perform suchfunctions as setting up an ID range, requesting a printout, directingthe system to perform cancer detection, etc.

Output interface 3 may enable the user to perform numerous types ofpost-processing manipulation of CAD results. FIG. 6 depicts a screenshot showing display of an image after CAD processing is complete. Imagedisplay 55 again contains an image, and the image is now a diagnosisimage that includes indications of potential abnormalities (e.g., asindicated by reference numeral 62) identified by the CAD processing.Potential abnormalities may be indicated not only by encirclement, asshown, but by any other means; for example, any shape (not just circles)may be used, arrows may be used, coloring may be used, etc.

The output interface shown in FIG. 6 also provides a display 61 forshowing comments provided by a user (e.g., a physician). The user mayenter such comments by any known means, for example, a keyboard, akeypad, a voice interface, a handwriting interface (tablet), a bar codereader, etc. Such comments 71 are shown, for example, in FIGS. 7 and 8.

The output interface 3 as shown in FIGS. 6-8 also includes a field 63having an image data display and a display of options that includesvarious input options. These input options permit, for example,printouts, image manipulation, and other post-processingfunctionalities. Image manipulation techniques implemented may include,for example, zoom in/out (i.e., magnification), masking, image rotation,etc. FIG. 8, in particular, shows a case in which a user has selectedmagnification button 82 in order to look at a particular region 81 ofimage display 55 in greater detail.

Another important function enabled by output interface 3 is what may betermed, “result editing.” Such result editing involves the userreviewing the CAD findings (as shown, e.g., by reference numeral 62 inFIG. 6) and making decisions regarding them. The user may agree ordisagree with the CAD findings, may delete any or all of them, and mayadd his or her own diagnostic findings to the image, using a pointingdevice (e.g., mouse, electronic pen, etc.). FIG. 7 depicts one exemplaryway in which such result editing might be implemented. In image display55, a CAD result that is confirmed (i.e., agreed with) by a user isshown having thin encirclement 72. A CAD result that is discarded (i.e.,disagreed with) by a user is shown having dashed encirclement 73.Finally, a user diagnosis, not detected by CAD processing, is shownhaving thick encirclement 74. Many other methods of such indications(e.g., different shapes, different colors, etc.) are possible. Theresults of the result editing can then be stored as the final results,along with any comments 71 the user may have.

The output interface shown in FIGS. 6-8 may enable additional functions.For example, a user may select a small region of interest and have theCAD system compute feature parameters on that small region. The user mayalso select a region of interest and have the CAD system re-examine thatregion. As with the initial output images from the CAD system, theresulting output images from such operations are then linked to outputinterface 3 for further examination by the user.

The invention has been described in detail with respect to preferredembodiments, and it will now be apparent from the foregoing to thoseskilled in the art that changes and modifications may be made withoutdeparting from the invention in its broader aspects. The invention,therefore, as defined in the appended claims, is intended to cover allsuch changes and modifications as fall within the true spirit of theinvention.

1. An input/output interface for a computer-aided diagnosis (CAD)system, the input/output interface comprising: an image display todisplay one or more indications of suspected abnormalities identified bythe CAD system; and a user interface to permit a user to edit at leastone of the indications of suspected abnormalities.
 2. The input/outputinterface of claim 1, wherein said user interface permits the user toperform at least one operation selected from the group consisting of:accepting an indication; rejecting an indication; and adding a newindication.
 3. The input/output interface of claim 2, wherein each ofsaid accepting, rejecting, and adding is shown in a different way onsaid image display.
 4. The input/output interface of claim 3, whereinsaid different way comprises a quality of a line indicating a suspectedabnormality.
 5. The input/output interface of claim 1, wherein said userinterface comprises at least one pointing device.
 6. A CAD systemcomprising the input/output interface of claim 1 and a computer systemrunning software for performing CAD processing.
 7. A method of providingan input/output interface for a computer-aided diagnosis (CAD) system,the method comprising: displaying one or more indications of suspectedabnormalities identified by the CAD system; and permitting a user toedit at least one of the indications of suspected abnormalities.
 8. Themethod of claim 7, wherein said permitting comprises at least one of thegroup consisting of: permitting the user to accept an indication;permitting the user to reject an indication; and permitting the user toadd a new indication.
 9. The method of claim 8, further comprising:showing each of accepting, rejecting, and adding an indication in adifferent way.
 10. The method of claim 9, wherein said different waycomprises a quality of a line indicating a suspected abnormality.
 11. Acomputer-readable medium containing software that, when executed by aprocessor, causes the processor to execute the method according to claim7.
 12. A CAD system comprising: a processor running CAD software; andthe computer-readable medium according to claim 11, saidcomputer-readable medium being coupled to said processor.
 13. A methodof reviewing results generated by a computer-aided diagnosis (CAD)system, the method comprising: viewing a display of one or moreindications of suspected abnormalities identified by the CAD system; andediting at least one of said indications of suspected abnormalities. 14.The method of claim 13, wherein said editing comprises: using a pointingdevice to select a suspected abnormality.
 15. The method of claim 13,wherein said editing comprises at least one of the group consisting of:accepting an indication; rejecting an indication; and adding a newindication.